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EXECUTIVE  SUMMARY 


The  Naval  Research  Laboratory’s  Remote  Sensing  Applications  Branch  has  developed  the 
Semi-Automated  Mesoscale  Analysis  System  (SAMAS),  a  prototype  image  analysis  system  for 
satellite  images  of  the  Gulf  Stream  region.  Previous  tests  demonstrated  that  SAMAS  shows  rudi¬ 
mentary  skill  in  automatically  locating  the  Gulf  Stream  and  its  associated  eddies.  Since  those  tests, 
SAMAS  has  been  updated  in  several  ways.  This  report  describes  the  performance  of  SAMAS 
Version  1.2,  a  newer  version  that  incorporates  some  new  techniques,  as  well  as  improvements  to 
some  old  ones. 

SAMAS  1.2  was  tested  by  applying  it  to  a  set  of  22  warmest-pixel  composite  multichannel  sea 
surface  temperature  (MCSST)  images  covering  February  through  June  1993,  which  were  also  ana¬ 
lyzed  by  a  human  expert.  The  human  analyses  were  regarded  as  “truth”  for  the  purposes  of  the 
statistical  analysis  of  the  test  results.  The  human  analyses  are  subjective  combinations  of  analyses 
from  the  Naval  Oceanographic  Office’s  Warfighting  Support  Center  and  information  from  inspec¬ 
tion  of  the  composite  images.  The  analyses,  both  human  and  automated,  include  North  Wall  positions 
and  eddy  definitions,  center  position  coordinates,  and  sizes. 

The  statistical  analysis  consists  of  tabulations  and  accuracy  measures.  For  eddies,  the  tabulations 
compare  the  number  of  eddies  found  by  SAMAS  versus  the  number  found  by  the  analyst,  and  the 
locations  and  sizes  of  the  eddies.  Accuracy  measures  include  position  errors  and  size  fractional 
errors,  their  first-  and  second-order  statistics  (means  and  standard  deviations),  as  well  as  minimum 
and  maximum  values.  The  Gulf  Stream  statistical  results  were  obtained  using  the  same  software 
that  was  used  to  evaluate  the  Gulfcast  and  Data  Assimilation  Research  and  Transition  (DART) 
numerical  models.  Statistics  include  information  on  the  longitude  range  spanned  by  each  Gulf 
Stream  determination,  mean  position  errors,  and  first-  and  second-order  statistics  of  the  latter. 

SAMAS  1.2  showed  some  improvement  over  the  earlier  version.  It  found  over  80%  of  the  rings 
the  analyst  found,  with  a  mean  position  error  of  approximately  22  km  and  a  mean  fractional  size 
error  of  0.165.  Gulf  Stream  mean  position  errors  were  in  the  range  of  40-45  km.  Gulf  Stream 
statistics  were  not  available  for  the  earlier  version,  so  a  comparison  is  not  possible. 


SEMI-AUTOMATED  MESOSCALE  ANALYSIS  SYSTEM, 
VERSION  1.2:  EVALUATION  TEST  RESULTS 


1.0  INTRODUCTION 

Infrared  (IR)  satellite  images  of  the  ocean  provide  surface  temperature  measurements  which 
can  be  used  either  to  supplement  local  measurements  at  various  depths  obtained  by  conventional 
oceanographic  techniques  or  to  provide  information  about  areas  of  the  ocean  where  data  from 
conventional  techniques  are  sparse.  Since  satellite  IR  images  often  depict  mesoscale  features  clearly, 
the  use  of  such  imagery  for  various  oceanographic  applications  is  expanding  rapidly.  However, 
present  interpretive  techniques  are  largely  manual,  require  significant  effort,  and  are  both  subjective 
in  nature  and  highly  dependent  upon  the  interpreter’s  skill.  With  the  proliferation  of  oceanographic 
analyses  that  use  satellite  data,  combined  with  decreasing  Navy  manpower  levels,  it  becomes  highly 
desirable  for  certain  applications  to  move  from  labor-intensive  manual  interpretation  toward  a 
capability  for  automated  image  interpretation. 

Over  the  past  several  years  the  Naval  Research  Laboratory’s  (NRL)  Remote  Sensing  Applications 
Branch  has  developed  a  prototype  image  analysis  system  that  shows  rudimentary  skill  in  automati¬ 
cally  locating  both  the  Gulf  Stream  and  eddies  in  IR  images  of  the  Gulf  Stream  region.  The  system 
is  known  as  the  Semi-Automated  Mesoscale  Analysis  System  (SAMAS).  Some  early  results  are 
given  by  Holyer  and  Peckinpaugh  (1990).  Since  that  earlier  report,  continuing  development  of  old 
techniques,  plus  the  addition  of  some  new  ones,  have  resulted  in  significant  changes  to  SAMAS. 

This  report  describes  evaluation  tests  of  the  latest  version  of  SAMAS,  Version  1.2,  and  provides 
a  statistical  analysis  of  the  test  results.  To  clarify  some  of  the  details,  an  outline  of  the  processing 
approach  that  SAMAS  employs  is  presented  here.  It  is  helpful  to  think  of  three  levels  of  analysis. 
The  lowest  level  consists  of  operations  that  focus  on  individual  image  pixels,  without  taking  advantage 
of  any  contextual  information.  SAMAS’  lowest  level  performs  image  segmentation.  Segmentation 
is  the  step  that  divides  an  image  into  regions,  which  in  some  cases  are  objects  in  the  images. 
Segmentation,  in  general,  can  be  edge-based  or  region-based.  The  output  of  the  lowest  level  is, 
therefore,  either  a  set  of  edges  or  a  set  of  regions  (the  regions  are  bounded  by  the  edges).  SAMAS 
uses  an  edge-based  segmenter  known  as  the  cluster  shade  edge  detector  (Holyer  and  Peckinpaugh  1989). 

The  detected  edges  are  passed  to  an  intermediate  level  which  performs  two  functions,  labeling 
and  feature  synthesis.  Labeling  consists  of  assigning  oceanographic  identities  to  the  edge  fragments 
created  by  the  segmenter.  Feature  synthesis  combines  edge  fragments  with  identical  labels  into 
continuous  features,  at  the  same  time  calculating  associated  parameters  such  as  position  and  radius. 
(These  two  steps  could  be  accomplished  with  region  fragments,  but  SAMAS  1.2  uses  an  edge 
segmenter.)  The  labeling  and  feature  synthesis  functions  are  a  mixture  of  conventional  image 
processing  and  artificial  intelligence,  i.e.,  some  algorithms  are  pixel-based  and  some  are  object- 
oriented  and  use  contextual  information.  SAMAS  1.2  offers  a  choice  of  two  labelers.  One  is  based 
on  nonlinear  probabilistic  relaxation  (Krishnakumar  et  al.  1990)  and  one  is  based  on  the  topography 
of  the  image  brightness  function  considered  as  a  surface  (Krishnamurthy  et  al.  1993).  The  feature 


1 


2 


Lybanon  and  Peckinpaugh 


synthesis  module  for  the  Gulf  Stream  uses  an  expansion  of  the  North  Wall  in  empirical  orthogonal 
functions  (EOF)  or  principal  components  (Molinelli  and  Flanigan  1987).  That  for  warm  and  cold 
rings  fits  circles  using  the  circular  Hough  transform  (Duda  and  Hart  1972). 

The  highest  level  consists  of  an  expert  system  that  describes  the  kinematics  of  mesoscale 
features  using  rules  about  the  time  evolution  of  eddies  (both  translational  motion  and  size  changes) 
and  a  neural  network  that  forecasts  the  coefficients  of  the  EOF  expansion  at  a  later  time  (Thomason 
1989;  Chase  and  Holyer  1993;  Lybanon  1994).  The  expert  system  has  two  main  functions  in 
SAMAS.  First,  the  forecast  feature  positions  can  provide  approximate  feature  locations  during 
periods  of  cloud  cover,  when  direct  observation  is  not  possible.  Second,  the  expert  system  can 
''update”  feature  positions  from  a  previous  analysis  to  provide  a  better  first  guess  for  the  relaxation 
labeler. 


2.0  SAMAS  AUTOMATED  PROCESSING 

The  test  images  are  a  set  of  MCSST  warmest-pixel  composites  that  cover  the  period  February 
through  June  1993.  Each  composite  consists  of  one  to  four  images,  spanning  a  1-  to  5-day  period. 
A  cloud  mask  was  computed  for  each  image,  and  a  composite  cloud  mask  was  created  from  the 
individual  cloud  masks  for  each  of  the  test  images.  The  method  used  to  create  the  cloud  mask  was 
developed  by  Gallegos  et  al.  (1993)  (see  Fig.  la  for  an  example  MCSST  warmest-pixel  composite 
image  with  cloud  mask).  We  acquired  the  oceanographic  analyses  for  the  north  Atlantic  Ocean 
covering  the  test  period  from  the  Warfighting  Support  Center  (WSC)  of  the  Naval  Oceanographic 
Office  (NAVOCEANO).  The  analyses  include  North  Wall  positions  and  eddy  definitions,  center 
position  coordinates,  and  sizes.  The  actual  days  of  the  WSC  analyses  do  not  correspond  exactly 
with  the  dates  of  our  image  data  (see  Table  1  for  image  dates  and  Fig.  2  for  WSC  dates).  For  each 
of  the  composite  images,  a  human  analyst  created  a  subjective  analysis  consisting  of  North  Wall 
positions  and  eddy  definitions,  based  on  the  composite  image  and  WSC  analyses  for  the  composite 
image  time  span.  These  will  be  referred  to  as  the  human  analysis. 

The  first  step  of  the  automated  processing  is  edge  creation,  using  the  cluster  shade  edge  detector 
(Holyer  and  Peckinpaugh  1989).  The  window  size  for  computing  the  gray-level  co-occurrence 
(GLC)  matrix  used  to  compute  the  image  of  cluster  shade  values  is  16  x  16  pixels,  and  the  delta 
X  and  y  values  are  zero.  For  every  overlapping  16-  x  16-pixel  window  of  the  image,  a  cluster  shade 
value  is  computed.  Using  these  values,  edges  are  defined.  The  zero-crossing  test  is  performed  as 
follows:  For  every  overlapping  3-  x  3-pixel  window  of  the  image  containing  the  cluster  shade 
values,  an  initial  zero-crossing  test  for  the  cluster  shade  values  is  done.  If  within  the  window  there 
is  a  value  greater  than  200  and  another  value  less  than  -200,  an  edge  is  found.  The  edge  is  denoted 
by  a  value  of  one  at  the  location  of  the  window’s  center  in  the  output  binary  edge  image,  to  be 
referred  to  henceforth  as  the  edge  image.  Once  this  initial  test  is  done,  multiple  passes  are  made 
through  the  cluster  shade  image  to  extend  these  edges  (up  to  30  passes  or  when  no  new  edges  are 
found).  Next,  for  every  overlapping  3-  x  3-pixel  window  of  the  cluster  shade  image,  a  test  is 
performed.  If  the  center  pixel  is  not  already  marked  as  an  edge,  and  any  of  the  other  pixels  in  the 
window  are  edge  pixels,  then  the  cluster  shade  values  of  the  window  are  tested.  If  there  is  a  value 
greater  than  100  and  another  value  less  than  -100,  an  edge  is  found  and  thus  marked  in  the  edge 
image. 

The  last  step  in  the  creation  of  the  edge  image  is  a  cleaning  step.  For  every  overlapping 
16-  X  16-pixel  window  of  the  edge  image,  the  following  test  is  performed.  If  the  border  of  the 
window  has  no  edge  marked,  but  edge  points  are  contained  within  the  window,  then  the  window 
is  "cleaned,”  i.e.,  edge  pixels  are  set  to  zero  (see  Fig.  lb  for  an  example  edge  image). 
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Fig.  1  _  (a)  MCSST  warmest-pixel  composite  for  April  20-24  with  composite  cloud  mask  displayed  over  the  image; 
(b)  cluster  shade  edges  displayed  over  the  image;  (c)  relaxation-labeled  edges  displayed  over  the  image  (yellow  for 
Gulf  Stream  North  Wall  edges,  red  for  eddy  edges);  and  (d)  CEOF  North  Wall  displayed  in  yellow,  Hough  eddies 
displayed  in  blue 


At  this  point  we  have  the  test  image,  cloud  mask,  edges,  and  human  analysis  for  each  of 
the  22  cases.  For  edge  labeling,  the  test  image,  cloud  mask,  and  a  previous  analysis  are  needed. 
With  the  exception  of  the  first  image  (in  temporal  sequence),  the  previous  analysis  will  be  the 
human  analysis  from  the  previous  image.  For  the  first  image,  a  WSC  analysis  will  serve  as 
the  previous  analysis.  The  next  step  is  to  propagate  the  previous  analysis  forward  in  time  to 
match  the  current  image  time.  The  oceanographic  expert  system  accomplishes  this  task  (Lybanon 
1990).  The  expert  system  depicts  expected  motions  and  size  changes  of  warm-  and  cold-core  rings, 
and  their  interaction  with  each  other  and  with  the  Gulf  Stream  in  the  northwest  Atlantic  Ocean.  The 
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Table  1  —  Dates  of  Images  Used  for  MCSST  Warmest  Composites, 
Notice  that  Some  Composites  Contain  Multiple  Images  from  a 
Single  Day. 


No. 

Date  Range 

Images  Used  to  Create  Composite 

1 

Mar  1-2 

Mar  1 

Mar  2 

Mar  2 

2 

Mar  8 . 12 

Mar  8 

Mar  8 

Mar  12 

3 

Mar  12-16 

Mar  12 

Mar  16 

4 

Mar  21-23 

Mar  21 

Mat  23 

5 

Mar  28-31 

Mar  28 

Mar  28 

Mar  31 

6 

Mar  31“Apr  4 

Mar  31 

Apr  2 

Apr  4 

7 

Apr  4-7 

Apr  4 

Apr  6 

Apr  7 

8 

Apr  7-11 

Apr  7 

Apr  10 

Apr  11 

Apt  11 

9 

Apr  18-20 

Apr  18 

Apr  19 

Apr  20 

10 

Apr  20-24 

Apr  20 

Apr  24 

11 

Apr  24-26 

Apr  24 

Apr  26 

Apr  26 

Apr  26 

12 

May  8-10 

May  8 

May  9 

May  9 

May  10 

13 

May  9-12 

May  9 

May  9 

May  10 

May  12 

14 

May  16-19 

May  16 

May  17 

May  18 

May  19 

15 

May  27-28 

May  27 

May  28 

16 

May  28-30 

May  28 

May  30 

17 

Jun  2-5 

Jun  2 

Jun  5 

Jun  5 

18 

Jun  9-12 

Jun  9 

Jun  12 

19 

Jun  13-14 

Jun  13 

Jun  14 

20 

Jun  16 

Jun  16 

Jun  16 

21 

Jun  21 

Jun  21 

22 

Jun  26-27 

Jun  26 

Jun  27 
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Fig.  2  —  Days  for  WSC  analyses  (marked  bold) 


domain  is  divided  into  nine  regions.  The  rules  that  describe  the  expected  behavior  of  the  warm-  and 
cold-core  rings  differs  in  each  region.  For  each  ring  in  the  analysis,  the  expert  system  calculates 
a  new  size  and  center  position  at  a  later  time.  The  expert  system  progresses  the  Gulf  Stream’s  North 
Wall  in  time  using  a  trained  neural  network  rather  than  rules  (Chase  and  Holyer  1993).  Refer  to 
Table  2  for  the  number  of  days  that  the  previous  analyses  are  projected  in  time.  Where  the  number 
of  days  is  given  as  zero,  the  analyses  were  not  processed  by  the  expert  system,  but  used  “as  is.” 

The  image,  corresponding  cloud  mask  and  edge  image,  along  with  the  previous  analysis, 
projected  to  the  image  time  as  needed,  are  used  by  the  edge-labeling  routine  to  create  a  new  labeled 
edge  image.  There  are  two  steps  in  executing  the  probabilistic  relaxation  labeling  algorithm 
(Krishnakumar  et  al.  1990).  In  the  first  step,  a  priori  probabilities  are  evaluated  with  the  help  of 
a  previous,  as-needed,  time-projected  analysis.  In  the  second  step,  these  a  priori  probabilities  are 
iteratively  updated  (relaxation)  until  a  consistent  labeling  is  reached.  The  resulting  labeled  edge 
image  contains  zero  values  at  no  edge  or  discarded  edges.  One  nonzero  value  is  used  to  represent 
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Table  2  —  Shown  for  Each  Test  Image  is  the  Previous  Analysis  for  Labeling  Images  and  Filling 
in  Missing  Eddies,  Along  with  the  Number  of  Days  Projected  by  the  Expert  System  for  the 
Analysis.  Also  Shown  is  the  Previous  Analysis  for  CEOF  Modes  Creation  and  Number  of  Days 
Projected  by  the  Expert  System.  All  Previous  Analyses  are  Created  by  the  Human  Analyst  Except 
Where  Specified  as  WSC. 


No. 

Image 

Previous  Analysis 
Used  for  Labeling 
and  Eddy  Fill-In 
for  Missing  Data 

No.  of  Days 
Projected 
Forward  by 
Expert  System 

Previous  Analysis 
Used  by  CEOF 
to  Create  Modes 

No.  of  Days 
Projected 
Forward  by 
Expert 
System 

1 

Mar  1-2 

Feb  27  (WSC) 

2 

Feb  27  (WSC) 

2 

2 

Mar  8-12 

Mar  1-2 

6 

Mar  1-2 

6 

3 

Mar  12-16 

Mar  8-12 

0 

Mar  1-2 

6 

4 

Mar  21-23 

Mar  12-16 

5 

Mar  12-16 

5 

5 

Mar  28-31 

Mar  21-23 

5 

Mar  21-23 

5 

6 

Mar  31-Apr  4 

Mar  28-31 

0 

Mar  28-31 

0 

7 

Apr  4-7 

Mar  31-Apr  4 

0 

Mar  28-31 

0 

8 

Apr  7-11 

Apr  4-7 

0 

Apr  4-7 

0 

9 

Apr  18-20 

Apr  7-11 

7 

Apr  7-11 

7 

10 

Apr  20-24 

Apr  18-20 

0 

Apr  7-11 

7 

11 

Apr  24-26 

Apr  20-24 

0 

Apr  7-11 

7 

12 

May  8-10 

Apr  24-26 

12 

Apr  24-26 

12 

13 

May  9-12 

May  8-10 

0 

Apr  24-26 

12 

14 

May  16-19 

May  9-12 

4 

May  9-12 

4 

15 

May  27-28 

May  16-19 

8 

May  16-19 

8 

16 

May  28-30 

May  27-28 

0 

May  27-28 

0 

17 

Jun  2-5 

May  28-30 

2 

May  28-30 

2 

18 

Jun  9-12 

Jun  2-5 

4 

Jun  2-5 

4 

19 

Jun  13-14 

Jun  9-12 

1 

Jun  9-12 

1 

20 

Jun  16 

Jun  13-14 

2 

Jun  13-14 

2 

21 

Jun  21 

Jun  16 

5 

Jun  16 

5 

22 

Jun  26—27 

Jun  21 

5 

Jun  21 

5 
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the  North  Wall  of  the  Gulf  Stream,  and  a  different  nonzero  value  is  assigned  to  each  eddy  (defined 
by  edges)  in  the  output  labeled  edge  image.  Figure  Ic  shows  an  example  of  a  labeled  edge  image. 

The  Gulf  Stream  North  Wall  positions  are  extracted  from  the  labeled  image  and  converted  from 
image-pixel  coordinates  to  latitude  and  longitude.  These  positions  are  then  input  to  the  Complex 
Empirical  Orthogonal  Function  (CEOF)  module  used  for  Gulf  Stream  formation  (Molinelli  and 
Flanigan  1987).  The  CEOF  module  also  uses  a  Gulf  Stream  to  create  a  mode  file.  This  Gulf  Stream 
must  be  of  good  quality,  i.e.,  the  full-length  stepwise  straight-line  interpolation  is  believable.  It  is 
used  to  “prime”  the  formation  of  the  current  Gulf  Stream.  Table  2  shows  images  and  the  Gulf 
Stream  used  to  create  the  required  mode  file.  The  number  of  days  used  by  the  expert  system  to 
project  the  human  analysis  forward  in  time  is  the  same  as  that  used  for  the  labeling.  When  a  Gulf 
Stream  proves  to  be  unusable  for  mode  file  creation,  older  previous  analyses  are  checked  until  a 
suitable  Gulf  Stream  is  found  (see  Fig.  Id  for  an  example  of  a  CEOF-generated  Gulf  Stream). 

The  eddy  edges  are  extracted  from  the  labeled  edge  image  to  form  an  eddy  edge  image.  These 
eddy  edges  are  dilated  to  double  width  and  then  input  to  the  eddy  detection  routine.  The  eddy 
detection  uses  a  modified  circular  Hough  transform  (Peckinpaugh  and  Holyer  1994).  Circular  fea¬ 
tures,  eddies,  are  defined  by  center  x,  y  position  and  pixel  radius  sizes.  These  values  are  converted 
to  latitude,  longitude  position  and  kilometer  radius.  The  Hough  transform  modification  biases  the 
results  toward  smaller  circular  features.  For  each  possible  radius,  an  accumulator  array  image  is 
created  (reasonable  eddy  size  for  our  region  is  50—133  km).  Each  element  in  the  accumulator  array 
image  contains  the  sum  of  points  making  up  the  circle  at  the  corresponding  x,  y  center  position  of 
the  original  image.  These  values  are  normalized  for  the  number  of  points  which  define  a  circle 
of  the  radius  size  that  corresponds  to  the  accumulator  array  image.  From  a  series  of  these  accumulator 
array  images,  the  best  (i.e.,  largest)  values  are  selected.  These  values  define  the  center  position  and 
size  values  assigned  to  the  eddy.  A  test  is  performed  to  eliminate  overlapping  of  eddies.  The  test 
uses  a  minimum  threshold  so  that  40%  of  the  circle  must  be  defined  to  detect  an  eddy.  Figure  Id 
shows  an  example  of  Hough  transform-defined  eddies. 

Due  to  cloud  cover  in  the  image,  some  eddies  may  not  be  visible  or  be  defined  by  clear  edges. 
For  this  reason,  the  Hough  transform  eddies  are  combined  with  eddies  from  the  previous  analysis. 
The  eddies  from  the  previous  analysis,  progressed  in  time  by  the  expert  system  as  needed,  are  used 
to  fill  in  eddies  not  defined  by  the  Hough  transform  (see  Table  2  for  previous  analysis  definitions). 
A  zero  value  in  the  “No.  of  days  projected  forward  by  Expert  System”  column  means  that  no  expert 
system  was  used  for  that  case.  Hough  transform  eddies  are  given  first  priority  where  there  is  any 
overlap  with  the  previous  analysis  eddies.  This  combined  eddy  list  defines  the  eddies  for  the  current 
image  (see  Fig.  Id  for  an  example  of  automated  analysis).  Note  that  all  eddies  for  this  case  were 
created  by  the  Hough  transform. 


3.0  STATISTICAL  ANALYSIS 

3.1  SAMAS  vs.  Human  Analysis  Eddies 

The  test  data  set  consisted  of  22  images.  However,  the  human  expert  did  not  find  eddies  in  nine 
of  the  images,  so  the  SAMAS  tests  employed  13  images  for  which  the  analyst  found  one  or  more 
eddies.  Let  E  =  the  number  of  eddies  found  by  the  expert  in  one  image,  and  let  5  =  E  +  iV  =  the 
number  of  eddies  found  by  SAMAS  in  one  image.  Table  3  shows  the  values  of  N  for  the  13  cases 
for  which  the  expert  found  at  least  one  eddy. 
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Table  3  —  Number  of 

Occurrences  of  Each  Value 
of  the  Difference  N  Between  the 
Number  of  Eddies  Found  by 
SAMAS  and  the  Number  Found 
by  the  Human  Expert 


N 

Number  of  Images 

-7 

1 

-5 

1 

-2 

3 

-1 

2 

0 

4 

+1 

2 

There  were  two  cases  in  which  (apparently)  the  same  eddy 
could  be  tracked  over  an  extended  period,  and  there  were  sig¬ 
nificant  discrepancies  between  the  expert  and  SAMAS  values 
of  both  position  and  size.  Those  two  cases  are  tabulated  in 
Table  4.  The  abbreviations  used  in  the  “Sources'’  columns  in  the 
table  are  as  follows:  EE  =  eddy  editor,  OC  =  (NAVOCEANO) 
Operational  Oceanography  Center,  ED  =  edge  detector,  and 
ES  =  expert  system. 

Table  5  gives  the  position  and  size  errors  for  each  date 
interval.  In  the  table,  “ground  truth”  values  are  those  found  by 
a  human  analyst,  and  “comparison”  values  are  from  SAMAS. 


3.2  Gulf  Stream  Error  Analysis 

3.2  J  Means  of  Evaluation 


Note:  For  =  0,  in  some  cases  SAMAS-produced  Gulf  Stream  North  Walls  are  compared 

the  SAMAS-found  eddies  and  the  with  those  produced  by  a  human  analyst  using  a  program 

analyst-found  eddies  were  not  developed  by  Geraldine  Gardner  of  Harvard  University,  origi- 

the  same  eddies  (i.e.,  there  were  nally  for  use  in  evaluating  the  Harvard  Gulfcast  model.  The 
significant  differences  in  their  Gulf  Streams  are  compared  over  a  specified  longitude 

geographical  positions).  range  by  interpolating  each  to  a  standard  grid.  The  error  is 

estimated  by  calculating  the  area  between  them  and  dividing 
that  area  by  the  arc  length  of  the  ground  truth  Gulf  Stream 
segment  within  that  longitude  range,  so  that  the  result  is  a  mean  error  expressed  in  kilometers.  That 
program  was  modified  slightly  by  Dan  Fox,  and  further  modified  by  Matthew  Lybanon  (both  of 
NRL)  to  read  the  SAMAS-format  Gulf  Stream  files.  However,  the  “core”  subroutine  that  performs 
the  error  calculation  (and  other  routines  called  by  it)  was  unchanged,  except  for  one  change  (by 
Fox)  to  compensate  for  the  different  sizes  of  a  degree  of  latitude  and  a  degree  of  longitude.  The 
original  Harvard  code  failed  to  compensate  for  this  difference. 


The  program  was  set  up  to  perform  the  calculation  over  four  longitude  ranges.  That  feature  was 
retained  and  used  as  described  below.  The  program  also  requires  the  data — both  ground  truth 
(analyst)  and  comparison  (SAMAS)  files — to  span  the  longitude  range  used  in  the  calculation. 
Since  each  file  covered  a  generally  different  longitude  range,  in  many  cases  it  was  necessary  to 
extrapolate  the  files  so  that  they  spanned  the  largest  longitude  range  used  in  the  analysis.  Table  6 
illustrates  that  extrapolation.  The  left  side  of  the  table  lists  the  original  longitude  limits  in  each  file, 
and  the  right  side  gives  the  limits  after  the  extrapolation.  In  every  case  except  one,  the  method  used 
was  linear  extrapolation  based  on  the  first  two  or  last  two  points  in  the  file.  The  exception  was  for 
the  June  21  “analyst”  Gulf  Stream,  which  was  very  short.  It  was  necessary  to  extrapolate  the 
western  end  from  -59.8077°  to  -80.0°  (a  distance  greater  than  the  original  length).  Two  points 
resulting  from  a  linear  least-squares  fit  to  the  first  six  points  in  the  original  file  were  necessary  to 
obtain  reasonable  results. 

The  most  straightforward  approach  is  to  compare  all  the  cases  over  the  same  longitude  ranges. 
However,  it  was  not  clear  that  this  approach  was  advisable  because  of  the  large  variations  between 
the  longitude  ranges  actually  covered  by  the  different  data  sets  (shown  in  Table  6).  The  approach 
adopted  for  the  analysis  is  a  compromise.  The  four  longitude  ranges  used  in  the  analysis  were  split 
into  two  pairs  of  ranges.  Two  “global”  ranges,  the  same  for  all  data  sets,  make  up  the  first  pair. 
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Table  4  —  Two  Cases  for  Which  There  were  Significant  Differences  Between  SAMAS- 

Derived  and  Expert-Derived  Eddies 


Eddy  A 


Ground  Truth 

Errors 

Sou 

rces 

Dates 

Lat. 

Lon. 

Rad. 

Pos.  (km) 

Fract.  Size 

Expert 

SAMAS 

24-26  Apr  93 

39.1454 

-68.419 

61.5132 

62.25707 

-0.01714 

EE 

ES 

16-19  May  93 

39.5446 

-68.651 

39.2824 

41.49992 

-0.01143 

EE 

ES 

27-28  May  93 

39.4251 

-69.910 

46.9616 

66.62056 

0.373079 

EE 

ED 

28-30  May  93 

39.3852 

-69.885 

47.2837 

20.10504 

0.732155 

EE 

ED 

2-5  Jun  93 

39.2854 

-69.910 

45.6201 

13.71933 

0.743262 

EE 

ED 

13-14  Jun  93 

39.3652 

-70.347 

44.7902 

20.70246 

0.539013 

EE 

ED 

Means: 

37.48406 

0.393157 

Eddy  B 

Ground  Truth 

Errors 

Soil 

trees 

Dates 

Lat. 

Lon. 

Rad. 

Pos.  (km) 

Fract.  Size 

Expert 

SAMAS 

20-24  Apr  93 

40.5100 

-65.570 

64.8200 

15.38535 

0.632211 

OC 

ED 

24-26  Apr  93 

40.5129 

-65.617 

73.9719 

14.94265 

0.231426 

EE 

ED 

8-10  May  93 

40.5326 

-65.180 

37.1211 

83.69115 

0.764086 

EE 

ED 

9-12  May  93 

40.4737 

-65.180 

37.4276 

35.02390 

0.632581 

EE 

ED 

16-19  May  93 

40.5914 

-65.566 

66.7426 

19.09353 

0.015390 

EE 

ED 

27-28  May  93 

40.1391 

-66.106 

60.6521 

0 

0 

EE 

EE 

28-30  May  93 

40.0404 

-66.183 

71.4185 

12.96360 

-0.000570 

EE 

ES 

13-14  Jun  93 

40.1588 

-65.951 

51.5543 

24.70246 

0.539013 

EE 

ED 

16  Jun  93 

40.2377 

-66.209 

48.4476 

34.42538 

-0.001430 

EE 

ES 

Means: 

26.69200 

0.312523 

Notes:  (Eddy  B)  For  the  27-28  May  93  data,  both  the  expert  and  SAMAS  values  were  input 
by  the  eddy  editor,  so  they  were  identical. 

(Both  eddies)  In  almost  every  case,  the  edge  detector  found  bigger  eddies  than  the 
expert.  This  was  also  true  for  other  eddies  in  the  test  cases. 
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Note  1-All  dates  are  in  1993. 

Note  2 -Eddy  center  latitude  and  longitude  are  in  degrees;  eddy  radius  is  in  kilometers. 

Note  3- A  “different  eddies”  notation  means  that  the  “comparison”  data  set  did  not  contain  an  eddy 
corresponding  to  the  one  in  the  “ground  truth”  data  set;  the  one  listed  is  the  comparison 
eddy  closest  to  the  ground  truth  eddy. 


Table  6  —  Gulf  Stream  File  Longitude  Limits 


SAM  AS  Version  1.2:  Evaluation  Test  Results 
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Note  1  -  All  dates  are  in  1993. 

Note  2  -  A  “same”  entry  in  an  “extrapolated”  column  means  that  the  value  is  the  same  as  the  one  in  the 
corresponding  “original”  column. 
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“Global  1,”  the  longer,  is  -79.8°  to  -54.0°;  “Global  2”  is  -74.6°  to  -63.6°.  The  values  were  based 
on  the  distributions  of  endpoint  longitudes.  Table  7  shows  the  longitude  ranges  used  as  the  second 
pair.  They  are  specific  to  each  data  set.  The  “Special  1”  range  is  always  longer  than  “Special  2.” 

The  Global  1  values  were  chosen  so  that,  on  the  average,  the  SAMAS  (generally  longer)  data 
ranges  prior  to  extrapolation  covered  the  interval.  Global  2  values  were  chosen  so  that,  again  on 
the  average,  both  SAMAS  and  analyst  data  ranges  prior  to  extrapolation  covered  the  interval. 
Hence,  Global  2  is  a  more  conservative  choice.  Special  1  and  Special  2  values  were  chosen  simi¬ 
larly,  uniquely  for  each  data  set.  As  a  result.  Special  2  is  a  more  conservative  choice  than  Special  1. 


5.2.2  Error  Statistics 

Table  8  shows  the  results  of  the  Gulf  Stream  mean  position  errors  for  all  four  longitude  ranges 
for  each  data  set,  as  well  as  the  overall  means  and  standard  deviations.  Global  2  and  Special  2 
results  have  comparable  mean  values,  and  because  of  the  way  the  ranges  were  chosen,  the  results 


Table  7  —  Gulf  Stream  Longitude  Ranges 
Specific  to  Each  Date  (“Special”  Values) 


Date 

Special  1 

Special  2 

Min. 

Max. 

Min. 

Max. 

Mar  8-12 

-77.5 

-59.6 

-72.2 

-61.7 

Mar  12-16 

-77.5 

-60.6 

-72.2 

-70.3 

Mar  21-23 

-77.5 

-60.3 

-72.4 

-65.5 

Mar  28-31 

-77.5 

-52.4 

-72.2 

-63.5 

Mar  31-Apr  4 

-74.8 

-60.4 

-74.4 

-68.4 

Apr  7-11 

-77.5 

-54.5 

-72.3 

-62.6 

Apr  18-20 

-77.5 

-58.55 

-72.3 

-58.60 

Apr  20-24 

-77.3 

-51.8 

-72.2 

-65.0 

Apr  24-26 

-77.5 

-56.7 

-72.3 

-60.4 

May  8-10 

-77.5 

-58.0 

-72.4 

-59.3 

May  9-12 

-77.7 

-58.5 

-72.1 

-60.5 

May  16-19 

-77.5 

-57.3 

-72.3 

-59.2 

May  27-28 

-77.5 

-56.9 

-72.2 

-62.2 

May  28-30 

-74.9 

-55.8 

-74.0 

-59.7 

Jun  2-5 

-77.7 

-55.0 

-72.4 

-65.9 

Jun  9-12 

-75.3 

-56.9 

-74.0 

-67.0 

Jun  13-14 

-77.4 

-55.0 

-72.2 

-65.0 

Jun  16 

-77.4 

-55.9 

-72.3 

-62.0 

Jun  21 

-69.9 

-54.2 

-59.8 

-58.9 

Jun  26-27 

-78.6 

-62.0 

-73.5 

-70.1 

Note  1-AIl  dates  are  in  1993. 

Note  2 -“Global”  longitude  ranges  are  -79.8  to 
-54.0  and  -74.6  to  -63.6. 
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Table  8  —  SAMAS  Gulf  Stream  Mean  Position  Errors 


Note  1-All  dates  are  in  1993. 

Note  2  -  Column  headings  for  position  errors  refer  to  longitude 
ranges  used  in  calculations. 

Note  3  -  Position  errors  are  in  kilometers. 

in  those  two  columns  are  judged  to  be  more  reliable.  It  is  probably  best  to  ignore  the  other  two 
columns. 

4.0  CONCLUSIONS 

This  study  quantifies  the  feasibility  of  automated  analysis  of  satellite  imagery  reported  by 
Holyer  and  Peckinpaugh  (1990)  by  testing  SAMAS  on  a  large  data  set  and  providing  a  statistical 
analysis  of  the  results.  SAMAS  1.2  found  over  80%  of  the  rings  the  human  expert  found,  with  a 
mean  position  error  of  approximately  22  km  and  a  mean  fractional  size  error  of  0.165.  These  figures 
compare  favorably  with  those  reported  for  the  earlier  version  of  SAMAS  (Holyer  and  Peckinpaugh 
1990).  This  study  also  presents  quantitative  information  on  the  accuracy  of  Gulf  Stream  North  Wall 
location  for  the  first  time.  Gulf  Stream  mean  position  errors  were  in  the  range  of  40-45  km. 
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